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a unit for operating the robot, and 

a display unit that can display the gray scale visual image and an image 
processing manipulation menu; and 
an image processing unit used for image processing; and 

said display unit displays an indication for generating or editing of the robot program and 
an indication for manipulation of image processing, together with the gray scale visual image of 
the robot simultaneously and allows a user to select between using the pendant for image 
processing teaching and robot teaching/control. 

REMARKS 

In the Office Action mailed February 12, 2003 the Examiner noted that claims 1-13 were 
pending. Claims 4, 8 and 1 1 have been amended, new claim 14 has been added and, thus, in 
view of the forgoing claims 1-14 remain pending for reconsideration which is requested. No new 
matter has been added. The Examiner's rejections are traversed below. 

On page 2 of the Action the rejected claims 1-7 and 11-13 under 35 U.S.C. section 112, 
paragraph 1 . Essentially, the Examiner indicates the reason for the need for the converting of 
image data stored in the memory is not understood, the term "intermediate image data" data is 
not understood and the reason for the converting of the intermediate data is not understood. 

The image data stored in the memory may be of a different gray scale resolution than 
what the pendant display can actually display. If this is the case, the data in the memory needs 
to be converted to the gray scale resolution of the pendant display. This is noted on page 25, 
line 21 to page 26, line 2 of the application specification. 

The term intermediate image data can be camera image data that has been processed, 
such as by binarization, edge enhancement, etc., before it is a final image or processed in a final 
stage. This is a well understood term in image processing for an intermediate image that is not 
the final version of an image. See, for example, 

(http://www.dpreview.com/learn/Glossary/DigitaiJmaging/JPEG_01.htm) where the term is used 
for an intermediate stage image that is in one of several stages of editing before being finally 
compressed. Because such an intermediate image may have it's gray scale resolution changed, 
it may need converting for the same reasons as the data in the memory, as discussed above. 

Withdrawal of the rejection of claims 1-7 and 1 1-13 is requested. 

On page 2 of the Action the rejected claims 1, 2, 4 and 8 under 35 U.S.C. section 112, 
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paragraph 1 . Essentially the Examiner indicates that the terms switching mode and 
superposition mode are not supported in the specification and not understood. The Examiner is 
requested to see page 7, lines 5-9 and page 9, lines 2-6 for the support for these terms in the 
specification. The passages, according to the understanding of the undersigned, explain that 
the display unit can be set for display of an image and the image processing manipulation 
(editing or generating a program) in a switching mode or a superposition mode, so that the 
image and program are either switched between or both displayed at the same time. 

Withdrawal of the rejection of claims 1,2,4 and 8 is requested. 

Page 3 of the Office Action rejects all claims under 35 U.S.C. § 103 over Tanabe and 
Jyumonji. 

The present invention is directed to a robot controller that includes image processing 
capability and a pendant used to control the robot or program (create or edit a program) the 
robot as well as including a display unit for image processing operations. The image processing 
capability includes a camera for capturing an image and processing for displaying the captured 
image on the pendant. As a display unit the pendant can thus display not only a program but 
also the image captured by the camera. The pendant, as a combined display unit and 
programming unit, can use a captured image as part teaching the robot. For example, the 
pendant can be used position a robot into a desired position, to display an image of the robot at 
that position and that image can be processed to program the robot to move to that position. In 
an image processing teaching mode the pendant can be used to specify an image to be 
processed for other purposes, such as position recognition. The pendant can also be used to 
control or manipulate the operations of the image processing, such as to initiate edge 
enhancement of a captured image. As a result, the present invention is a system that allows the 
operator to control/manipulate/program a robot, display an image from a camera, manipulate 
image processing operations and/or confirm image processing. The prior art does not teach or 
suggest such. 

The Examiner cites Jyumonji for its alleged teaching of a obtaining image data from a 
camera. 

The Examiner cites Tanabe for allegedly teaching or suggesting the remainder of the 
features of the claims. 

In particular the Examiner notes that Tanabe discusses a pendant 10 displaying "a state 
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of the robot" and "input data" and cites Tanabe col. 3, at about lines 22-31 . In making this 
comparison the Examiner appears to consider the robot state as including the input data and 
equates the input data with the image captured by the camera and displayed on the pendant in 
the present invention. Tanabe discusses several different types of inputs, some associated with 
the pendant and some with another device, such as a personal computer. These inputs include: 
input of a command, such as a teaching command, input into the personal computer via a jog 
input device; input via a touch panel of the pendant display for controlling operations of the 
robot; and graphical input through though a graphical user interface supplied to the personal 
computer. The discussion of possible types of input data in Tanabe, even input data that is not 
destined for the pendant, does not include an image captured by a camera. That is, Tanabe 
says nothing and suggests nothing about having a pendant that can be used for teaching/control 
of a robot and/or teaching an image processing system and that displays an image supplied by a 
camera, as in the claimed invention. Jyumonji adds nothing to Tanabe with respect to this 
feature of the claimed invention. 

It is submitted that the present invention is patentably distinguishable over the prior art for 
the above discussed reason. 

In making the rejection of a number of the claims, starting on page 3, the Examiner used 
claim 4 as a model. On page 3 of this rejection of claim 4, the Examiner particularly equated the 
"input data" discussed in the phrase "... the liquid display unit 13 for displaying information of the 
robot and input data ... (see col. 3, about lines 26-32) with the image converted into gray scale of 
claim 4. Tanabe discusses nothing about and does nolm enti on at.all conv.e iting-^anjmaqgjnto^ 
gray^scalaJmage-orJakiagj^ 

image or p rocessing intermediate images resulti ng from a camera image. Tanabe does not 
discuss the integration of a display concerned with robot manipulation and robot operational 
programming or a display concerned with image processing and manipulation of image 
processing as is the case with the present claimed invention. Tan abe is involved jA^ith-Usjng a 
personal computer to simplifiuaboLroaaifiuiatio^ contrast, the 

present invention is designed to enable an operator to perform effective manipulation associated 
within^ 

THe^re^|tjs^a_pej^^ thej^bot, contr^innage 

^rocess^^^ operations. Nothing in the prior art of 

Tanabe and/or Jyumonji teach or suggest this. 
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With respect to claim 3, on page 6 of the Office Action the Examiner alleges that Tanabe 
at col. 3, lines 45-47 intuitively or impliedly teaches the feature of claim 3 related to superposing 
geometric graphics on the displayed camera image. This feature of the present invention is 
more particularly described at page 27, lines 20-22, page 28, lines 12-13 and page 30, lines 20- 
24. There is nothing in Tanabe that teaches or suggests such graphic superposition operations. 

It is submitted that the invention of claims 1-13 distinguishes over the prior art and 
withdrawal of the rejection is requested. 

New claim 14 emphasizes the above discussed features of the invention. Nothing in the 
prior art teaches or suggests such. It is submitted that new claim 14 distinguishes over the prior 
art. 

It is submitted that the claims satisfy the requirements of 35 U.S.C. 112. It is further 
submitted that the claims are not taught, disclosed or suggested by the prior art. The claims are 
therefore in a condition suitable for allowance. An early Notice of Allowance is requested. 

If any further fees, other than and except for the issue fee, are necessary with respect to 
this paper, the U.S.P.T.O. is requested to obtain the same from deposit account number 19- 
3935. 



Respectfully submitted, 



STAAS & HALSEY LLP 




Registration No. 30,358 



700 Eleventh Street, NW, Suite 500 
Washington, D.C. 20001 
(202) 434-1500 
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VERSION WITH MARKINGS TO SHOW CHANGES MADE 

IN THE CLAIMS: 

Please AMEND the following claims: 

1 . (Previously Amended) An image processing apparatus for a robot which is built in a 
robot controller, and has a portable teaching pendant connected thereto, comprising: 

a unit for fetching an image from a camera; 

memory storing image data from the camera or intermediate image data obtained in a 
stage of image processing; and 

a unit for converting image data from the camera, the image data from the camera stored 
in the memory, or the intermediate image data into a gray scale or a color scale, wherein: 

said teaching pendant comprises a display unit and a unit used for manipulation for 
image processing and display of the converted image data; and 

said display unit displays an image and indication for manipulation of image processing 
simultaneously, or allows a user to select either a switching mode or a superposition mode. 

2. (Previously Amended) An image processing apparatus for a robot which is designed 
independent of the robot controller, and has a portable teaching pendant connected thereto, 
comprising: 

a unit for fetching an image from a camera; 

memory storing image data from the camera or intermediate image data obtained in a 
stage of image processing; and 

a unit for converting image data from the camera, the image data from the camera stored 
in the memory, or the intermediate image data into a gray scale or a color scale, wherein: 

said teaching pendant comprises a display unit and a unit used for manipulation for 
image processing and display of the converted image data; and 

said display unit displays an image and indication for manipulation of image processing 
simultaneously, or allows a user to select either a switching mode or a superposition mode, 

wherein the robot is designed independently of the robot controller. 

3. (Previously Amended) The apparatus according to claim 1, further comprising: 

a unit for displaying and superposing geometric graphics on the image displayed on the 
display unit in accordance with the operation procedure of image processing and specifying an 
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image processing with respect to tlie image. 

4. (Currently Amended) An image processing apparatus for a robot which is built in a 
robot controller, comprising: 

a unit for fetching an image from a camera; 

memory which stores image data from the camera or intermediate image data obtained 
in a stage of image processing; and 

a unit for converting image data from the camera, the image data from the camera stored 
in the memory, or intermediate image data into a gray scale or a color scale, wherein: 

a portable teaching pendant is connected to said robot controller through a cable; and 

said teaching pendant comprises a unit for generating or editing a robot program, a unit 
for operating the robot, and a display unit, and can display on the display unit [an image 
converted into the gray scale] the converted image , and comprises a unit used for manipulation 
for image processing; and 

said display unit displays, indication for generating or editing of the robot a program and 
indication for manipulation of image processing, together with an image simultaneously, or 
allows a user to select either a switching mode or a superposition mode. 

5. (Original) The apparatus according to claim 4, further comprising: 

a unit for displaying and superposing geometric graphics on the image displayed on the 
display unit in accordance with the operation procedure of the image processing and specifying 
an image processing with respect to the image. 

6. (Previously Amended) The apparatus according to claim 1, wherein 

a part of the operation unit of the teaching pendant is configured by a touch panel. 

7. (Original) The apparatus according to claim 4, further comprising 

a unit for incorporating an instruction to process an image into a program of a robot. 

8. (Currently Amended) A method, comprising: 
fetching an image from a camera; 

storing image data from the camera or intermediate image data obtained in a stage of 
image processing; and 
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converting image data from the camera, the image data from the camera stored in the 
memory, or the intermediate image data into a gray scale or a color scale; and 
displaying the converted image data on a teaching pendant, 

wherein said displaying displays the image data and indication for manipulation for image 
processing simultaneously, or allows a user to enter a switching mode or a superposition mode. 

9. (Previously Added) An apparatus, comprising: 

a image processor receiving an image from a camera; 
a robot control unit controlling a robot; and 

a portable teaching pendant displaying the image and allowing a user to control the robot 
using the robot control unit. 

10. (Previously Added) A method, comprising: 
receiving an image from a camera; and 

displaying the image on a portable teaching pendant; and 
controlling the robot using the portable teaching pendant. 

11 . (Currently Amended) The apparatus according to claim [1] 2, further comprising: 

a unit for displaying and superposing geometric graphics on the image displayed on the 
display unit in accordance with the operation procedure of image processing and specifying an 
image processing with respect to the image. 

12. (Previously Added) The apparatus according to claim 2, wherein 

a part of the operation unit of the teaching pendant is configured by a touch panel. 

13. (Previously Added) The apparatus according to claim 4, wherein 

a part of the operation unit of the teaching pendant is configured by a touch panel. 

Please ADD the following claims: 

14. (New) An image processing apparatus for a robot which is built in a robot controller, 
comprising: 

a unit for fetching a visual image of the robot from a camera; 
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memory which stores the visual image from the camera or obtained in a stage of image 
processing; and 

a unit for converting the visual image into a gray scale visual image; 
a portable teaching pendant connected to said robot controller through a cable; and said 
teaching pendant comprising: 

a unit for generating or editing a robot program, 

a unit for operating the robot, and 

a display unit that can display the gray scale visual image and an image 
processing manipulation menu; and 
an image processing unit used for image processing; and 

said display unit displays an indication for generating or editing of the robot program and 
an indication for manipulation of image processing, together with the gray scale visual image of 
the robot simultaneously and allows a user to select between using the pendant for image 
processing teaching and robot teaching/control. 
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